fix(gp): bless live response oracles for 9 unblessed scripts + gate future regrowth - #229
Merged
Merged
Conversation
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
mikemcdougall
had a problem deploying
to
staging
September 10, 2026 21:39 — with
GitHub Actions
Failure
|
You have reached your Codex usage limits for security reviews. Please try again later. |
3 tasks
mikemcdougall
had a problem deploying
to
staging
September 14, 2026 07:15 — with
GitHub Actions
Failure
mikemcdougall
had a problem deploying
to
staging
September 14, 2026 07:20 — with
GitHub Actions
Failure
mikemcdougall
had a problem deploying
to
staging
September 14, 2026 07:35 — with
GitHub Actions
Failure
mikemcdougall
had a problem deploying
to
staging
September 14, 2026 07:45 — with
GitHub Actions
Failure
mikemcdougall
had a problem deploying
to
staging
September 14, 2026 08:18 — with
GitHub Actions
Failure
mikemcdougall
had a problem deploying
to
staging
September 14, 2026 08:25 — with
GitHub Actions
Failure
mikemcdougall
had a problem deploying
to
staging
September 14, 2026 08:39 — with
GitHub Actions
Failure
mikemcdougall
had a problem deploying
to
staging
September 14, 2026 08:47 — with
GitHub Actions
Failure
…moving canary The blocking live GP smoke lane consumed the moving nightly-aot tag and an unpinned honua-server seed checkout, so a commit's verdict depended on what honua-server merged that day. - pinned (push/pull_request): image digest + seed revision of the 2026.1-rc.2 governed candidate (honua-server@7ba4226, sha256:dd50cd81...), checked against the pulled image's digest and revision label; seed checked out at that exact revision. Documented promotion path in the job comment. - canary (daily schedule / dispatch mode=canary): resolves nightly-aot to a digest + revision and seeds from it; every step is non-fatal so the check-run never concludes failure; the verdict creates/updates the "honua-gp moving canary drift" issue. - a verdict step separates environment-startup-failure, gp-correctness-failure, eval-incomplete and not-executed from passed, records smoke-identity.json, and a missing compose fixture now fails instead of skipping green. - the persistent HONUA_GP_EVAL_BASE_URL target is honoured on manual dispatch only; the canary has its own concurrency group. Closes #201
mikemcdougall
force-pushed
the
fix/202-gp-response-oracles
branch
from
September 14, 2026 09:58
846cbc5 to
96ec5c8
Compare
mikemcdougall
had a problem deploying
to
staging
September 14, 2026 09:58 — with
GitHub Actions
Failure
…uture regrowth Live smoke graded no response oracle for 9 of 25 supported honua-gp eval scripts (Describe x2, ListFields x3, InsertCursor, UpdateCursor x2), so a response-parsing regression in schema introspection or write-cursor round trips would pass the lane silently as an "unblessed" pass. Blessing the two write-cursor scripts against a fresh seed surfaced a real bug rather than a missing oracle: honua_gp.da._values_for_row/_extract_oid only recognized a feature's .attributes, but the live SDK's honua_sdk.models.QueryFeature exposes GeoJSON-shaped .properties/.id instead -- SearchCursor/UpdateCursor silently returned None for every field (OID included) against a real server. Stub-mode CI never caught this because the eval stub's _StubFeature happens to have .attributes. Fixed both helpers to read the real feature shape (preferring the SDK's already-resolved .id for OID@, since the seed's object-id field is lower-case objectid and never matched the existing OBJECTID/oid/FID key list), added regression tests against the actual QueryFeature type, and re-blessed against a fresh stack -- update_cursor_close_status now genuinely proves an update round-trip (update_count: 1) instead of masking the bug behind a vacuous zero. Describe/ListFields (schema fetch) and InsertCursor were already correct; those got a schema/apply-edits fingerprint oracle. MakeTableView makes no server round trip at all (a client-side session alias registration) -- no response to fingerprint, so it goes in a new UNBLESSED_ALLOWLIST.json with the reason instead of a silent pass. run_eval.py now fails a live-mode supported script that has no response oracle and no UNBLESSED_ALLOWLIST.json entry, so this set cannot regrow unnoticed (verified: an emptied allow-list correctly fails make_table_view.py with the new reason). Closes #202
…view The write-cursor oracles recorded applyEdits counts only, and update_cursor_delete_closed was blessed to a vacuous delete_count 0: an earlier script had already archived every CLOSED row, so a broken deleteRow passed. make_table_view still passed as an allow-listed "unblessed" script. - insert_cursor_append_rows reads the inserted object ids back and records the persisted rows. - update_cursor_close_status / update_cursor_delete_closed insert their own fixture row, scope the edit to it by name, and record the value left behind / that the deleted row is gone. They no longer depend on script order or on a previous run's rows. - make_table_view applies a where clause and records the rows the server returns through the view (the five 'active' seed rows). - The allow-list is gone: a live supported script without a response oracle fails, and a unit test fails the stub lane when a supported golden has none. - The generator now owns these scripts' oracles, so regenerating no longer drops them.
mikemcdougall
force-pushed
the
fix/202-gp-response-oracles
branch
from
September 14, 2026 10:06
96ec5c8 to
3b46f68
Compare
mikemcdougall
had a problem deploying
to
staging
September 14, 2026 10:06 — with
GitHub Actions
Failure
Contributor
Author
|
Rebuilt head |
mikemcdougall
had a problem deploying
to
staging
September 14, 2026 10:29 — with
GitHub Actions
Failure
mikemcdougall
had a problem deploying
to
staging
September 14, 2026 11:13 — with
GitHub Actions
Failure
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #228 (its current head
7309939), because trunk'sephemeral-server-smokeis red. The unpinned seed insertsservices.max_record_count, but thenightly-aotimage schema has no such column, so the seed fails with psql exit 3. Once #228 merges, this diff shrinks to its two commits.Problem (#202)
ephemeral-server-smokegraded no response oracle for 9 of the 25 supportedhonua-gpeval scripts:describe_segments[_fields]list_fields_segments[_filtered|_wildcard]insert_cursor_append_rowsupdate_cursor_close_statusupdate_cursor_delete_closedmake_table_viewA supported script with no oracle counted as a silent "unblessed" pass. So a regression in
Describe/ListFieldsor in the write cursors could pass this lane unnoticed.--require-supported-pass-rate 1.0did not help, because unblessed scripts counted as passing.Changes
Commit 1 (the earlier revision, rebased onto #228). Blessing the cursor scripts exposed a real bug.
honua_gp.daread only.attributes, but the live SDK'sQueryFeatureexposes.properties/.id. As a result, every liveSearchCursor/UpdateCursorfield read, OID included, returnedNone. This commit:QueryFeature;Describe/ListFields;Commit 2 (this revision). Commit 1 still left three gaps:
update_cursor_delete_closedwas blessed todelete_count: 0. An earlier script had already archived every CLOSED row, so it deleted nothing, and a brokendeleteRowwould have passed.make_table_viewstill passed through an allow-list as "unblessed".Commit 2 closes them:
Insert.
insert_cursor_append_rowsreads back the object ids the server assigned and records the rows that persisted.Update and delete.
update_cursor_close_statusandupdate_cursor_delete_closedeach insert their own fixture row and scope the edit to it by name. They then read back:Neither depends on script order or on rows left by an earlier run.
Table view.
make_table_viewappliesstatus = 'active'and records the rows the server returns through the view (GetCount+SearchCursor).Gate. The allow-list is removed. In live mode, a supported script without a response oracle now always fails. A new unit test,
test_every_supported_script_has_a_response_oracle, fails the stub lane as soon as a supported golden lacks one.Generator.
_generate_scripts.pynow owns these four scripts' oracles. Before this change, regenerating the scripts silently dropped the hand-edited emit lines.Per-script decision
All 9 scripts have an observable result, so the unblessed set is now empty.
describe_segments,describe_segments_fields,list_fields_segmentsobjectid, shapePoint, SRID 4326honua.layer_fieldsfor layer 0 intests/seed/client-compat-v1.sqlat 7ba4226list_fields_segments_filtered/_wildcardstatusinsert_cursor_append_rowsadd_count 3and the persisted rows[CLOSED, Side Rd], [OPEN, Elm Ave], [OPEN, Main St]update_cursor_close_statusupdate_count 1, the updated row reads[ARCHIVED, Close Status Rd],closed_remaining 0update_cursor_delete_closeddelete_count 1,deleted_inserted_row true,rows_remaining 0make_table_viewview_count 5, rows alpha, epsilon, eta, gamma, iota (allactive)status = 'active'rowsEvidence
All live runs used the pinned candidate:
ghcr.io/honua-io/honua-server@sha256:dd50cd81…with the seed at7ba4226. The stack was started with the workflow's own compose override (migrate first, then seed, then the SpatialJoin join layer).Fresh stack, committed goldens, at head
3b46f68:honua-gp eval [live]: 52/52 passed (100%); supported 27/27 (100%).Bless run on a fresh stack: only the four scripts changed in this revision produced new goldens. The other 48 matched their committed goldens.
Rerun against the already-mutated database: all four changed scripts pass again. The older seed-pinned count scripts drift there as expected, as
docs/golden-eval.mdalready documents.Negative control: I reverted only the
.propertiesfallback in_attrs_for_featureand ran just the four changed scripts live. The result wassupported 0/4:update_cursor_delete_closedgotdelete_count 0,rows_remaining 1. The old count-only oracle blessed exactly that as a pass.make_table_viewgot[[null, null] × 5].I restored the file afterwards and the worktree was clean.
Unit tests:
python -m pytest packages/honua-gp/tests -qgives 281 passed, 9 skipped.Other checks: stub eval 52/52;
python -m honua_gp._cli matrix --check packages/honua-gp/docs/compatibility-matrix.mdpasses.Out of scope: ArcGIS Pro output parity is not claimed. The oracles are server-observed values on the client-compat seed, as described in
docs/golden-eval.md.Cross-links
Closes #202